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1. 


Introduction 


In  [2]  Wielandt  and  Hofftnan  proved  a  theorem  on  the  eigenvalues  of 
normal  matrices  which  is  of  considerable  importance  in  the  error  analysis 
of  eigenvalue  algorithms  based  on  the  use  of  unitary  transformations 
[U, 5].  Their  proof  was  very  elegant  and  was  based  on  the  use  of  linear 
programming  techniques.  In  [5]  Wilkinson  gave  an  elementary  proof  in 
the  case  when  the  matrices  are  Hermitian,  which  was  based  on  an  earlier 
proof  due  to  Givens  [1].  This  proof  did  not  extend  easily  to  the  general 
case.  Here  we  give  an  elementary  proof  for  the  general  case  which 
applies  immediately  to  a  generalization  of  the  Wielandt -Hof  flnan  theorem 
due  to  Kahan  [3].  Not  surprisingly  the  proof  involves  techniques  which 
are  familiar  in  the  area  of  linear  programming  but  no  direct  appeal  is 
made  to  results  from  that  field. 


2.  The  Basic  Theorem 

The  proof  depends  on  a  theorem  which  is  not  directly  concerned  with 
normal  matrices.  Before  stating  this  theorem  we  give  two  definitions. 

DEFINITION  1.  The  set  of  n  elements  a,  a  J  of 

-  l,ix  2,  ig  n,in 

an  nxn  matrix  A  is  called  a  diagonal  of  A  if  i^, ig,  .  ..,in  is 
a  permutation  of  the  integers  1,2, ...,n  .  If  i^  =  j  (j  =  1,  ...,n) 
then  we  have  the  principal  diagonal. 

DEFINITION  2.  A  matrix  X  is  called  a  doubly  stochastic  matrix  if 

x.  >  0  and  fx.,  =  f  x„  =1  (J  =  1, . . .,n)  i.e.,  all  row 

***  i=l 

and  column  runs  are  unity. 
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THEOREM  1.  If  P  is  a  real  matrix  such  that  the  sum  of  the 
elements  on  the  principal  diagonal  is  not  greater  than  the  sum 
of  the  elements  on  any  other  diagonal,  and  X  is  any  doubly 
stochastic  matrix,  then  S(X)  s  T.Z  p^jXij  is  a  minimum  when 


Proof.  Yiic  minimum  is  attained,  possibly  for  many  different  X  . 
Let  us  choose  M  to  be  a  minimizing  doubly  stochastic  matrix  having 
the  maximum  number  of  zero  off-diagonal  elements.  We  shall  show  that 
all  its  off-diagonals  must  be  zero.  For  suppose  that  this  is  not  true. 


Let  x. 


11,12 


be  a  non-zero  off-diagonal.  Then  x.  .  <  1  and  hence 

V  2 


there  is  a  non-zero  element  x.  .  (say)  in  row  i»  .  If  i,  /  i0 

i2,13  3  d 

then  similarly  there  is  a  non-zero  element  x.  .  in  row  i,  . 

3 

Continue  in  this  way  until  we  reach  an  x.  .  for  which  i  equals 

i  i  >  f  _  m 

m-1  m 


some  earlier  i.  .  Let  x  be  the  smallest  of  the 
k 


Lve  elements 


x.  .  ,x. 

V  1k+l  Vl'  V  2 


* '  *  ,,xi  ,,i 
m-17  k 


Construct  a  matrix  Y  such  that 


y.  =  x.  ,  +  x  ,  s  =  k,k+l, ...,m-l 

x,i  i,i  77 

s  s  s'  s 


y  =  x  ,  -  x  ,  s  =  k,k+l, . .  .,m-l 

iii,-  ill,. 

s7  s+1  s  s+1 


y  =  x  otherwise. 

1 J  1«J 


Then  Y  is  clearly  a  doubly  stochastic  matrix  and 


(2.1) 


(2.2) 


(2.3) 


2L 


EEpii^h  -EEp,  ^,,  =  *  E  Pi  i  ■  E  pi  i 
ij  ij  ij  ij  £k  is^is  s%  VV: 


■ijAij 


(2.4) 


The  expression  in  brackets  cannot  be  positive  since  otherwise  by  replacing 

the  elements  p.  .  in  the  principal  diagonal  by  the  elements 
i  >  i 


we  could  obtain  a  smaller  diagonal  sum.  Hence 


V  s+1 


sEEPi/u 


But  Y  is  clearly  a  doubly  stochastic  matrix  and  it  has  at  least  one 
more  off-diagonal  zero  than  X  ,  contradicting  the  hypothesis.  Hence 
all  off-diagonal  elements  of  X  must  be  zero,  i.e.,  X  -  I  . 

An  exactly  analogous  theorem  holds  when  the  principal  diagonal 
has  the  maximum  sum. 


3.  The  Wielandt-Hofftnan  Theorem 

THEOREM  2.  If  A  and  B  are  normal  matrices  and  C  -  A  -  B  ,  and 

if  a.  and  b^  are  the  eigenvalues  of  A  and  B  arranged  so 
n  2 

that  ^  lai-t>il  a  fn^n^num  ^or  possible  orderings,  then 

^|ai-bij2  <  ||c||p  .  (||C|jp  =  the  Frobenius  norm  of  C)  (3.1) 


Proof.  Since  A  and  B  are  noiroal  there  exist  unitary  Q  and  Q2 


such  that 


A  =  Q1  diagfa^  ,  B  =  Q2  diagCb^Qg  • 


(3.2) 


(Note  then  we  are  free  to  prescribe  the  ordering  of  the  and  b^ 
and  we  choose  the  ordering  which  gives  £  |a^-bj^  a  minimum  value. 
Hence 

A  -  B  =  Qx  diag(a.)Q*J  -  Q2  diag^Qg  =  C  .  (3.3) 

giving 

diag(ai)Q^  Q2  -  Q2  diag(bi)  =  qJcQ2  .  (3.b) 

Writing  Q  =  Q1  Q2  >  a  unitary  matrix,  we  have 

||diag(a.)Q  -  Q  diag(b.)j|p  =  |jc||p  (3.5) 


since  the  Frobenius  norm  is  unitarily  invariant.  Hence 


lvbi 


J1  *  ij 
Now  the  matrix  P  with  p 


2  2 

=  Hen; 


=  a.  -  b  .  I 
ij  1  i  J 


(3.6) 


is  real  and  from  the  ordering 


of  the  a.  and  d.  its  principal  diagonal  is  minimal.  Further,  since 

2 

Q  ic  unitary,  the  matrix  Z  with  z.  .  =  |q.  .|  is  a  doubly  stochastic 
matrix.  Hence  by  Theorem  1  and  equation  (3.6) 


t  lai-bii2  <  EEK  -bJ2  k 


id 


(3.7) 


and  the  result  is  proved. 

When  A  and  B  are  Hermitian,  the  a^  and  are  real,  and  it 
is  easy  to  prove  that  the  orderings  >  a2  >  •  •  •  >  an  >  b]_  >  b2  -  '  *  *  —  bn 

give  the  minimal  value.  In  fact,  returning  to  Theorem  1  in  the  case  when 

p 

D  =  (a  -b  )  with  a.  and  b.  real  and  monotonically  ordered,  the 
ij  i  j  i  i 

proof  is  much  simpler.  For  if  X  has  a  non-zero  off  diagonal  element 


U 


in  row  1  or  column  1  it  must  have  at  least  one  such  in  both.  Suppose 

x,  and  x are  non-zero  and  x  is  the  smaller.  If  we  increase 

ir  sr 

x.,  and  x  _  by  x  and  diminish  x,  and  x  ,  by  x  the  sum  is 
-l-L  sr  lr  si 

changed  by 

x[(a1-b1)2+  (ag  - by)2  -  (ax  - bp)2  -  (afl  -b]_)]2  =  x(ax  -  ag)(br  -b^  <  0  ( 

Hence  continuing  in  this  way  the  minimizing  X  has  no  non-zero  off-diagonal 
elements  in  row  1  or  column  1,  and  continuing  again  the  minimizing  X 
is  I  .  (Notice  we  do  not  even  have  to  show  that  for  this  P  ,  the 
principal  diagonal  is  minimal;  this  emerges  from  the  proof.) 

4.  Generalization  of  the  Wlelandt-Hofflnan  Theorem 

A  generalization  of  the  Wielandt -Hoffman  Theorem  which  is  of 
practical  importance  is  the  following. 

THEOREM  3.  If  X  is  an  nxr  matrix  with  orthonormal  columns, 

A  is  an  nyn  normal  matrix,  B  is  an  rxr  normal  matrix 
and  R  an  nxr  matrix  is  defined  by 

AX  -  XB  =  R  ,  (4.1) 

if  the  eigenvalues  a^  (i  =  1,  ...,n)  of  A  and  b.^  (i  -  1,  ...,r) 

of  B  are  ordered  so  that  |  a .  -  b .  | 2  is  a  minimum,  then 

i=l  1  1 

EK  -»/  <  II 8 lip  •  (‘*•2) 

i=l 

2  l/  2  2 

A  weaker  result  with  |]  R  ||F  replaced  by  2  '  ||  R  ||F  was  given  by 
Wilkinson  in  [5l  and  the  result  itself  by  Kahan  [3]. 
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Notice  we  are  interested  only  in  the  selection  and  ordering  of  the 


relevant  r  of  the  a^  to  be  associated  with  the  .  Writing 

A  =  Q1  diag(a.)Q^  ,  B  =  diag^fc”  (^-3) 

with  the  prescribed  ordering  of  the  a^  and  b^  ,  we  have 
!!diag(a.)Q  -  Q  diag(b.)^  =  ||Q*  R  Q2|||  =  !'  R 
where  Q  is  an  nxr  matrix  with  ortho-normal  columns.  Hence 


(M) 


Let  Y  =  [Q  |  Z  ]  be  an  nxn  unitary  matrix  given  by  the  completion 
of  Q  ;  then  if 


pij  =  I  ai  -  b j  1 2  (j  <  r)  ,  Pi(.  =  0  (,i  >  r)  .  (b.G) 


£  Eh  -  h/M 

,1=1  i=l  J 


(*.7) 


and  from  the  definition  of  the  ordering  of  the  a^  and  ,  the  diagonal 
of  P  is  minimal.  Hence  by  Theorem  1  and  Equation  (U.5) 


£pu  =  fK-bi|24  |1iai-bji  i^r 


=  I!  R 


(*.8) 


This  theorem  is  of  practical  value  when  r  orthonormal  approximate 
eigenvectors  x^,  ...,xr  are  known  corresponding  to  alleged  eigenvalues 
•  •  •>  |ir  •  If 


6 


‘i  ’  ^iXi  =  ri 


(i  =  1, •  •  *»r) 


(^•9) 


AX  -  X  diag(^i)  =  R 


(U.10) 


with  an  obvious  notation,  and  diag^)  is  the  matrix  B  of  Theorem  3. 
This  then  states  that  there  exist  r  eigenvalues  a^,  ...,a  of  A 


such  that 


£  (ai  ■  ^i> 


2  -  ii  r  iip 


(4.n) 


Notice  that  the  ^  can  include  multiple  or  pathologically  chic 
eigenvalues.  The  result  is  well  known  when  r  =  1  and  the 
Wielandt-Hofflnan  theorem  corresponds  to  the  case  r  =  n.  We  observe 
that  by  using  less  than  r  of  the  alleged  eigenvectors  we  can  obtain 
results  of  the  type  (4.11)  corresponding  to  any  s  ( C  r)  of  the 
approximate  eigenvalues. 


*1W 
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